<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
<meta name="generator" content="Doxygen 1.12.0"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>NeuZephyr: nz::nodes::calc::MatMulNode Class Reference</title>
<link rel="icon" href="NZ_logo2.png" type="image/x-icon" />
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr id="projectrow">
  <td id="projectlogo"><img alt="Logo" src="NZ_logo2.png"/></td>
  <td id="projectalign">
   <div id="projectname">NeuZephyr
   </div>
   <div id="projectbrief">Simple DL Framework</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.12.0 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function() { codefold.init(0); });
/* @license-end */
</script>
  <div id="navrow1" class="tabs">
    <ul class="tablist">
      <li><a href="index.html"><span>Main&#160;Page</span></a></li>
      <li><a href="pages.html"><span>Related&#160;Pages</span></a></li>
      <li><a href="namespaces.html"><span>Namespaces</span></a></li>
      <li class="current"><a href="annotated.html"><span>Classes</span></a></li>
      <li><a href="files.html"><span>Files</span></a></li>
    </ul>
  </div>
  <div id="navrow2" class="tabs2">
    <ul class="tablist">
      <li><a href="annotated.html"><span>Class&#160;List</span></a></li>
      <li><a href="classes.html"><span>Class&#160;Index</span></a></li>
      <li><a href="inherits.html"><span>Class&#160;Hierarchy</span></a></li>
      <li><a href="functions.html"><span>Class&#160;Members</span></a></li>
    </ul>
  </div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function(){ initResizable(false); });
/* @license-end */
</script>
<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><b>nz</b></li><li class="navelem"><a class="el" href="namespacenz_1_1nodes.html">nodes</a></li><li class="navelem"><a class="el" href="namespacenz_1_1nodes_1_1calc.html">calc</a></li><li class="navelem"><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html">MatMulNode</a></li>  </ul>
</div>
</div><!-- top -->
<div id="doc-content">
<div class="header">
  <div class="summary">
<a href="#pub-methods">Public Member Functions</a> &#124;
<a href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node-members.html">List of all members</a>  </div>
  <div class="headertitle"><div class="title">nz::nodes::calc::MatMulNode Class Reference</div></div>
</div><!--header-->
<div class="contents">

<p>Represents a matrix multiplication operation node in a computational graph.  
 <a href="#details">More...</a></p>
<div class="dynheader">
Inheritance diagram for nz::nodes::calc::MatMulNode:</div>
<div class="dyncontent">
<div class="center"><img src="classnz_1_1nodes_1_1calc_1_1_mat_mul_node__inherit__graph.png" border="0" usemap="#anz_1_1nodes_1_1calc_1_1_mat_mul_node_inherit__map" alt="Inheritance graph"/></div>
<map name="anz_1_1nodes_1_1calc_1_1_mat_mul_node_inherit__map" id="anz_1_1nodes_1_1calc_1_1_mat_mul_node_inherit__map">
<area shape="rect" title="Represents a matrix multiplication operation node in a computational graph." alt="" coords="5,80,198,107"/>
<area shape="rect" href="classnz_1_1nodes_1_1_node.html" title="Base class for nodes in a neural network or computational graph." alt="" coords="43,5,160,32"/>
<area shape="poly" title=" " alt="" coords="104,48,104,80,99,80,99,48"/>
</map>
<center><span class="legend">[<a href="graph_legend.html">legend</a>]</span></center></div>
<div class="dynheader">
Collaboration diagram for nz::nodes::calc::MatMulNode:</div>
<div class="dyncontent">
<div class="center"><img src="classnz_1_1nodes_1_1calc_1_1_mat_mul_node__coll__graph.png" border="0" usemap="#anz_1_1nodes_1_1calc_1_1_mat_mul_node_coll__map" alt="Collaboration graph"/></div>
<map name="anz_1_1nodes_1_1calc_1_1_mat_mul_node_coll__map" id="anz_1_1nodes_1_1calc_1_1_mat_mul_node_coll__map">
<area shape="rect" title="Represents a matrix multiplication operation node in a computational graph." alt="" coords="5,80,198,107"/>
<area shape="rect" href="classnz_1_1nodes_1_1_node.html" title="Base class for nodes in a neural network or computational graph." alt="" coords="43,5,160,32"/>
<area shape="poly" title=" " alt="" coords="104,48,104,80,99,80,99,48"/>
</map>
<center><span class="legend">[<a href="graph_legend.html">legend</a>]</span></center></div>
<table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a id="pub-methods" name="pub-methods"></a>
Public Member Functions</h2></td></tr>
<tr class="memitem:a493c723e88870bdc46c9b30b74b1e173" id="r_a493c723e88870bdc46c9b30b74b1e173"><td class="memItemLeft" align="right" valign="top">&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="#a493c723e88870bdc46c9b30b74b1e173">MatMulNode</a> (<a class="el" href="classnz_1_1nodes_1_1_node.html">Node</a> *input_left, <a class="el" href="classnz_1_1nodes_1_1_node.html">Node</a> *input_right)</td></tr>
<tr class="memdesc:a493c723e88870bdc46c9b30b74b1e173"><td class="mdescLeft">&#160;</td><td class="mdescRight">Constructor to initialize a <code><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html" title="Represents a matrix multiplication operation node in a computational graph.">MatMulNode</a></code> for matrix multiplication.  <br /></td></tr>
<tr class="separator:a493c723e88870bdc46c9b30b74b1e173"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a4d1ec1a90036ff16358c5f83123bac67" id="r_a4d1ec1a90036ff16358c5f83123bac67"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="#a4d1ec1a90036ff16358c5f83123bac67">forward</a> () override</td></tr>
<tr class="memdesc:a4d1ec1a90036ff16358c5f83123bac67"><td class="mdescLeft">&#160;</td><td class="mdescRight">Forward pass for the <code><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html" title="Represents a matrix multiplication operation node in a computational graph.">MatMulNode</a></code> to perform matrix multiplication.  <br /></td></tr>
<tr class="separator:a4d1ec1a90036ff16358c5f83123bac67"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:ab644a874feb6a620ad31d37ca20525fd" id="r_ab644a874feb6a620ad31d37ca20525fd"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="#ab644a874feb6a620ad31d37ca20525fd">backward</a> () override</td></tr>
<tr class="memdesc:ab644a874feb6a620ad31d37ca20525fd"><td class="mdescLeft">&#160;</td><td class="mdescRight">Backward pass for the <code><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html" title="Represents a matrix multiplication operation node in a computational graph.">MatMulNode</a></code> to propagate gradients.  <br /></td></tr>
<tr class="separator:ab644a874feb6a620ad31d37ca20525fd"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="inherit_header pub_methods_classnz_1_1nodes_1_1_node"><td colspan="2" onclick="javascript:dynsection.toggleInherit('pub_methods_classnz_1_1nodes_1_1_node')"><img src="closed.png" alt="-"/>&#160;Public Member Functions inherited from <a class="el" href="classnz_1_1nodes_1_1_node.html">nz::nodes::Node</a></td></tr>
<tr class="memitem:a687ee9c34eb61f8f28caa201ca42696e inherit pub_methods_classnz_1_1nodes_1_1_node" id="r_a687ee9c34eb61f8f28caa201ca42696e"><td class="memItemLeft" align="right" valign="top">virtual void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1nodes_1_1_node.html#a687ee9c34eb61f8f28caa201ca42696e">print</a> (std::ostream &amp;os) const</td></tr>
<tr class="memdesc:a687ee9c34eb61f8f28caa201ca42696e inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="mdescLeft">&#160;</td><td class="mdescRight">Prints the type, data, and gradient of the node.  <br /></td></tr>
<tr class="separator:a687ee9c34eb61f8f28caa201ca42696e inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a9b85913e12422bb4ac2fff483427bb47 inherit pub_methods_classnz_1_1nodes_1_1_node" id="r_a9b85913e12422bb4ac2fff483427bb47"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1nodes_1_1_node.html#a9b85913e12422bb4ac2fff483427bb47">dataInject</a> (Tensor::value_type *data, bool grad=false) const</td></tr>
<tr class="memdesc:a9b85913e12422bb4ac2fff483427bb47 inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="mdescLeft">&#160;</td><td class="mdescRight">Injects data into a relevant tensor object, optionally setting its gradient requirement.  <br /></td></tr>
<tr class="separator:a9b85913e12422bb4ac2fff483427bb47 inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a609f1730085dd1d31e0ddcbbae48a065 inherit pub_methods_classnz_1_1nodes_1_1_node" id="r_a609f1730085dd1d31e0ddcbbae48a065"><td class="memTemplParams" colspan="2">template&lt;typename Iterator &gt; </td></tr>
<tr class="memitem:a609f1730085dd1d31e0ddcbbae48a065 inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="memTemplItemLeft" align="right" valign="top">void&#160;</td><td class="memTemplItemRight" valign="bottom"><a class="el" href="classnz_1_1nodes_1_1_node.html#a609f1730085dd1d31e0ddcbbae48a065">dataInject</a> (Iterator begin, Iterator end, const bool grad=false) const</td></tr>
<tr class="memdesc:a609f1730085dd1d31e0ddcbbae48a065 inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="mdescLeft">&#160;</td><td class="mdescRight">Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.  <br /></td></tr>
<tr class="separator:a609f1730085dd1d31e0ddcbbae48a065 inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:af8b4bab3271df92ca1f0914f7a97b1e8 inherit pub_methods_classnz_1_1nodes_1_1_node" id="r_af8b4bab3271df92ca1f0914f7a97b1e8"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1nodes_1_1_node.html#af8b4bab3271df92ca1f0914f7a97b1e8">dataInject</a> (const std::initializer_list&lt; Tensor::value_type &gt; &amp;data, bool grad=false) const</td></tr>
<tr class="memdesc:af8b4bab3271df92ca1f0914f7a97b1e8 inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="mdescLeft">&#160;</td><td class="mdescRight">Injects data from a std::initializer_list into the output tensor of the <a class="el" href="classnz_1_1nodes_1_1_node.html" title="Base class for nodes in a neural network or computational graph.">Node</a>, optionally setting its gradient requirement.  <br /></td></tr>
<tr class="separator:af8b4bab3271df92ca1f0914f7a97b1e8 inherit pub_methods_classnz_1_1nodes_1_1_node"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table>
<a name="details" id="details"></a><h2 class="groupheader">Detailed Description</h2>
<div class="textblock"><p>Represents a matrix multiplication operation node in a computational graph. </p>
<p>The <code><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html" title="Represents a matrix multiplication operation node in a computational graph.">MatMulNode</a></code> class performs matrix multiplication between two input tensors. It implements the matrix multiplication operation in the forward pass, and propagates the gradients during the backward pass. This node is typically used to represent fully connected layers or other linear algebraic operations in a neural network or computational graph. The node now leverages Tensor Cores for efficient half-precision matrix multiplication, improving performance during forward and backward passes.</p>
<p>Key features:</p><ul>
<li><b>Forward Pass</b>: The <code><a class="el" href="#a4d1ec1a90036ff16358c5f83123bac67" title="Forward pass for the MatMulNode to perform matrix multiplication.">forward()</a></code> method computes the matrix multiplication of two input tensors and stores the result in the <code>output</code> tensor. The computation is accelerated using Tensor Cores with half-precision (FP16) to speed up matrix multiplication operations.</li>
<li><b>Backward Pass</b>: The <code><a class="el" href="#ab644a874feb6a620ad31d37ca20525fd" title="Backward pass for the MatMulNode to propagate gradients.">backward()</a></code> method propagates the gradients from the output tensor to the input tensors using the chain rule of calculus.</li>
<li><b>Shape Check</b>: The constructor ensures that the number of columns in the left input tensor matches the number of rows in the right input tensor, as required for matrix multiplication.</li>
</ul>
<p>This class is part of the <code><a class="el" href="namespacenz_1_1nodes.html" title="Contains classes and functionality for nodes in a neural network or computational graph.">nz::nodes</a></code> namespace and is used for matrix operations in a computational graph.</p>
<dl class="section note"><dt>Note</dt><dd><ul>
<li>The left input tensor's number of columns must match the right input tensor's number of rows.</li>
<li>The matrix multiplication operation in this node uses Tensor Cores for faster computation using half-precision floating-point arithmetic (FP16).</li>
</ul>
</dd></dl>
<h3><a class="anchor" id="autotoc_md90"></a>
Usage Example:</h3>
<div class="fragment"><div class="line"><span class="comment">// Example 1: Creating and using a MatMulNode</span></div>
<div class="line">InputNode input1({3, 2}, <span class="keyword">true</span>);  <span class="comment">// Create the first input node with shape {3, 2}</span></div>
<div class="line">input1.output-&gt;fill(1.0f);  <span class="comment">// Fill the tensor with value 1.0</span></div>
<div class="line"> </div>
<div class="line">InputNode input2({2, 3}, <span class="keyword">true</span>);  <span class="comment">// Create the second input node with shape {2, 3}</span></div>
<div class="line">input2.output-&gt;fill(2.0f);  <span class="comment">// Fill the tensor with value 2.0</span></div>
<div class="line"> </div>
<div class="line"><a class="code hl_function" href="#a493c723e88870bdc46c9b30b74b1e173">MatMulNode</a> matmul_node(&amp;input1, &amp;input2);  <span class="comment">// Create a MatMulNode using the two input nodes</span></div>
<div class="line">matmul_node.forward();  <span class="comment">// Perform the forward pass: output = input1 * input2</span></div>
<div class="line">matmul_node.backward();  <span class="comment">// Perform the backward pass: propagate gradients</span></div>
<div class="line"> </div>
<div class="line">std::cout &lt;&lt; <span class="stringliteral">&quot;Output: &quot;</span> &lt;&lt; *matmul_node.output &lt;&lt; std::endl;  <span class="comment">// Print the output tensor</span></div>
<div class="ttc" id="aclassnz_1_1nodes_1_1calc_1_1_mat_mul_node_html_a493c723e88870bdc46c9b30b74b1e173"><div class="ttname"><a href="#a493c723e88870bdc46c9b30b74b1e173">nz::nodes::calc::MatMulNode::MatMulNode</a></div><div class="ttdeci">MatMulNode(Node *input_left, Node *input_right)</div><div class="ttdoc">Constructor to initialize a MatMulNode for matrix multiplication.</div><div class="ttdef"><b>Definition</b> <a href="_nodes_8cu_source.html#l00148">Nodes.cu:148</a></div></div>
</div><!-- fragment --><dl class="section see"><dt>See also</dt><dd><a class="el" href="#a4d1ec1a90036ff16358c5f83123bac67" title="Forward pass for the MatMulNode to perform matrix multiplication.">forward()</a> for the <a class="el" href="#a4d1ec1a90036ff16358c5f83123bac67" title="Forward pass for the MatMulNode to perform matrix multiplication.">forward</a> pass computation method. </dd>
<dd>
<a class="el" href="#ab644a874feb6a620ad31d37ca20525fd" title="Backward pass for the MatMulNode to propagate gradients.">backward()</a> for the <a class="el" href="#ab644a874feb6a620ad31d37ca20525fd" title="Backward pass for the MatMulNode to propagate gradients.">backward</a> pass gradient propagation method.</dd></dl>
<dl class="section author"><dt>Author</dt><dd>Mgepahmge (<a href="https://github.com/Mgepahmge">https://github.com/Mgepahmge</a>)</dd></dl>
<dl class="section date"><dt>Date</dt><dd>2024/11/29 </dd></dl>

<p class="definition">Definition at line <a class="el" href="_nodes_8cuh_source.html#l01060">1060</a> of file <a class="el" href="_nodes_8cuh_source.html">Nodes.cuh</a>.</p>
</div><h2 class="groupheader">Constructor &amp; Destructor Documentation</h2>
<a id="a493c723e88870bdc46c9b30b74b1e173" name="a493c723e88870bdc46c9b30b74b1e173"></a>
<h2 class="memtitle"><span class="permalink"><a href="#a493c723e88870bdc46c9b30b74b1e173">&#9670;&#160;</a></span>MatMulNode()</h2>

<div class="memitem">
<div class="memproto">
      <table class="memname">
        <tr>
          <td class="memname">nz::nodes::calc::MatMulNode::MatMulNode </td>
          <td>(</td>
          <td class="paramtype"><a class="el" href="classnz_1_1nodes_1_1_node.html">Node</a> *</td>          <td class="paramname"><span class="paramname"><em>input_left</em></span>, </td>
        </tr>
        <tr>
          <td class="paramkey"></td>
          <td></td>
          <td class="paramtype"><a class="el" href="classnz_1_1nodes_1_1_node.html">Node</a> *</td>          <td class="paramname"><span class="paramname"><em>input_right</em></span>&#160;)</td>
        </tr>
      </table>
</div><div class="memdoc">

<p>Constructor to initialize a <code><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html" title="Represents a matrix multiplication operation node in a computational graph.">MatMulNode</a></code> for matrix multiplication. </p>
<p>This constructor initializes an <code><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html" title="Represents a matrix multiplication operation node in a computational graph.">MatMulNode</a></code> which performs matrix multiplication between the outputs of two input nodes. It ensures that the shapes of the two input tensors are compatible for matrix multiplication. Specifically, the number of columns of the left input tensor must match the number of rows of the right input tensor. If the shapes do not match, an exception is thrown. The constructor also initializes the <code>output</code> tensor with the appropriate shape based on the input tensors and sets the <code>requires_grad</code> flag based on the input tensors' gradient tracking requirements.</p>
<dl class="params"><dt>Parameters</dt><dd>
  <table class="params">
    <tr><td class="paramname">input_left</td><td>A pointer to the first input node. Its <code>output</code> tensor is used for the matrix multiplication. </td></tr>
    <tr><td class="paramname">input_right</td><td>A pointer to the second input node. Its <code>output</code> tensor is used for the matrix multiplication.</td></tr>
  </table>
  </dd>
</dl>
<p>The constructor checks that the number of columns in the left input tensor (<code>input_left-&gt;output-&gt;shape()[1]</code>) matches the number of rows in the right input tensor (<code>input_right-&gt;output-&gt;shape()[0]</code>), as required for matrix multiplication. The output tensor is created with the shape <code>(input_left-&gt;output-&gt;shape()[0], input_right-&gt;output-&gt;shape()[1])</code>, and the <code>requires_grad</code> flag is set to <code>true</code> if either of the input tensors requires gradients.</p>
<dl class="exception"><dt>Exceptions</dt><dd>
  <table class="exception">
    <tr><td class="paramname">std::invalid_argument</td><td>If the shapes of the input tensors are not compatible for matrix multiplication.</td></tr>
  </table>
  </dd>
</dl>
<dl class="section note"><dt>Note</dt><dd><ul>
<li>The left input tensor's column count must match the right input tensor's row count for matrix multiplication.</li>
<li>The constructor ensures that the output tensor has the correct shape to hold the result of the matrix multiplication.</li>
<li>The <code>requires_grad</code> flag for the output tensor is set based on the gradient requirements of the input tensors.</li>
</ul>
</dd></dl>
<dl class="section author"><dt>Author</dt><dd>Mgepahmge (<a href="https://github.com/Mgepahmge">https://github.com/Mgepahmge</a>)</dd></dl>
<dl class="section date"><dt>Date</dt><dd>2024/11/29 </dd></dl>

<p class="definition">Definition at line <a class="el" href="_nodes_8cu_source.html#l00148">148</a> of file <a class="el" href="_nodes_8cu_source.html">Nodes.cu</a>.</p>

</div>
</div>
<h2 class="groupheader">Member Function Documentation</h2>
<a id="ab644a874feb6a620ad31d37ca20525fd" name="ab644a874feb6a620ad31d37ca20525fd"></a>
<h2 class="memtitle"><span class="permalink"><a href="#ab644a874feb6a620ad31d37ca20525fd">&#9670;&#160;</a></span>backward()</h2>

<div class="memitem">
<div class="memproto">
<table class="mlabels">
  <tr>
  <td class="mlabels-left">
      <table class="memname">
        <tr>
          <td class="memname">void nz::nodes::calc::MatMulNode::backward </td>
          <td>(</td>
          <td class="paramname"><span class="paramname"><em></em></span></td><td>)</td>
          <td></td>
        </tr>
      </table>
  </td>
  <td class="mlabels-right">
<span class="mlabels"><span class="mlabel">override</span><span class="mlabel">virtual</span></span>  </td>
  </tr>
</table>
</div><div class="memdoc">

<p>Backward pass for the <code><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html" title="Represents a matrix multiplication operation node in a computational graph.">MatMulNode</a></code> to propagate gradients. </p>
<p>The <code><a class="el" href="#ab644a874feb6a620ad31d37ca20525fd" title="Backward pass for the MatMulNode to propagate gradients.">backward()</a></code> method computes the gradients of the input tensors with respect to the output tensor for the matrix multiplication operation. During the backward pass, the gradients of the output tensor are propagated back to the two input tensors. The gradient computation follows the chain rule of calculus.</p>
<p>Specifically:</p><ul>
<li>For the left input tensor (<code>A</code>), the gradient is computed as <code>dA = dC * B^T</code>, where <code>dC</code> is the gradient of the output tensor and <code>B^T</code> is the transpose of the right input tensor.</li>
<li>For the right input tensor (<code>B</code>), the gradient is computed as <code>dB = A^T * dC</code>, where <code>A^T</code> is the transpose of the left input tensor and <code>dC</code> is the gradient of the output tensor.</li>
</ul>
<p>These gradients are computed on the GPU using CUDA kernels (<code>GeneralMatrixMul</code>), which parallelize the matrix operations.</p>
<dl class="section note"><dt>Note</dt><dd><ul>
<li>The gradients for both input tensors are computed only if they require gradients (i.e., <code>requiresGrad()</code> is true).</li>
<li>The gradients are computed using the transposes of the input matrices and propagated through the network.</li>
<li>The <code>GeneralMatrixMul</code> kernel is used for efficient gradient computation on the GPU.</li>
</ul>
</dd></dl>
<dl class="section see"><dt>See also</dt><dd><a class="el" href="#a4d1ec1a90036ff16358c5f83123bac67" title="Forward pass for the MatMulNode to perform matrix multiplication.">forward()</a> for the <a class="el" href="#a4d1ec1a90036ff16358c5f83123bac67" title="Forward pass for the MatMulNode to perform matrix multiplication.">forward</a> pass computation method.</dd></dl>
<dl class="section author"><dt>Author</dt><dd>Mgepahmge (<a href="https://github.com/Mgepahmge">https://github.com/Mgepahmge</a>)</dd></dl>
<dl class="section date"><dt>Date</dt><dd>2024/11/29 </dd></dl>

<p>Implements <a class="el" href="classnz_1_1nodes_1_1_node.html#a0a9ecbaa3d790ba38e8218aca7837fd0">nz::nodes::Node</a>.</p>

<p class="definition">Definition at line <a class="el" href="_nodes_8cu_source.html#l00169">169</a> of file <a class="el" href="_nodes_8cu_source.html">Nodes.cu</a>.</p>
<div class="dynheader">
Here is the call graph for this function:</div>
<div class="dyncontent">
<div class="center"><img src="classnz_1_1nodes_1_1calc_1_1_mat_mul_node_ab644a874feb6a620ad31d37ca20525fd_cgraph.png" border="0" usemap="#aclassnz_1_1nodes_1_1calc_1_1_mat_mul_node_ab644a874feb6a620ad31d37ca20525fd_cgraph" alt=""/></div>
<map name="aclassnz_1_1nodes_1_1calc_1_1_mat_mul_node_ab644a874feb6a620ad31d37ca20525fd_cgraph" id="aclassnz_1_1nodes_1_1calc_1_1_mat_mul_node_ab644a874feb6a620ad31d37ca20525fd_cgraph">
<area shape="rect" title="Backward pass for the MatMulNode to propagate gradients." alt="" coords="5,5,198,48"/>
<area shape="rect" href="namespacenz_1_1data.html#ac8d64dd271e9a2e50682e733bd14ec19" title="Transposes a tensor with a valid tensor type." alt="" coords="246,13,379,40"/>
<area shape="poly" title=" " alt="" coords="198,24,230,24,230,29,198,29"/>
</map>
</div>

</div>
</div>
<a id="a4d1ec1a90036ff16358c5f83123bac67" name="a4d1ec1a90036ff16358c5f83123bac67"></a>
<h2 class="memtitle"><span class="permalink"><a href="#a4d1ec1a90036ff16358c5f83123bac67">&#9670;&#160;</a></span>forward()</h2>

<div class="memitem">
<div class="memproto">
<table class="mlabels">
  <tr>
  <td class="mlabels-left">
      <table class="memname">
        <tr>
          <td class="memname">void nz::nodes::calc::MatMulNode::forward </td>
          <td>(</td>
          <td class="paramname"><span class="paramname"><em></em></span></td><td>)</td>
          <td></td>
        </tr>
      </table>
  </td>
  <td class="mlabels-right">
<span class="mlabels"><span class="mlabel">override</span><span class="mlabel">virtual</span></span>  </td>
  </tr>
</table>
</div><div class="memdoc">

<p>Forward pass for the <code><a class="el" href="classnz_1_1nodes_1_1calc_1_1_mat_mul_node.html" title="Represents a matrix multiplication operation node in a computational graph.">MatMulNode</a></code> to perform matrix multiplication. </p>
<p>The <code><a class="el" href="#a4d1ec1a90036ff16358c5f83123bac67" title="Forward pass for the MatMulNode to perform matrix multiplication.">forward()</a></code> method computes the matrix multiplication between the two input tensors using CUDA, and stores the result in the <code>output</code> tensor. The matrix multiplication is performed using the <code>GeneralMatrixMul</code> kernel on the GPU, which efficiently computes the product of the two matrices in parallel.</p>
<p>This method is called during the forward pass of the neural network. It calculates the matrix product of the left input tensor (<code>inputs[0]</code>) and the right input tensor (<code>inputs[1]</code>), and stores the result in the <code>output</code> tensor. The shape of the <code>output</code> tensor is determined by the number of rows in the left input tensor and the number of columns in the right input tensor.</p>
<dl class="section note"><dt>Note</dt><dd><ul>
<li>The kernel <code>GeneralMatrixMul</code> performs the matrix multiplication using parallel computation on the GPU.</li>
<li>The matrix multiplication is performed as <code>M = A * B</code>, where <code>A</code> is the left input tensor and <code>B</code> is the right input tensor.</li>
<li>The block size (<code>TILE_SIZE</code>) and grid size are chosen to ensure efficient GPU parallelization of the operation.</li>
</ul>
</dd></dl>
<dl class="section see"><dt>See also</dt><dd><a class="el" href="#ab644a874feb6a620ad31d37ca20525fd" title="Backward pass for the MatMulNode to propagate gradients.">backward()</a> for the <a class="el" href="#ab644a874feb6a620ad31d37ca20525fd" title="Backward pass for the MatMulNode to propagate gradients.">backward</a> pass gradient propagation method.</dd></dl>
<dl class="section author"><dt>Author</dt><dd>Mgepahmge (<a href="https://github.com/Mgepahmge">https://github.com/Mgepahmge</a>)</dd></dl>
<dl class="section date"><dt>Date</dt><dd>2024/11/29 </dd></dl>

<p>Implements <a class="el" href="classnz_1_1nodes_1_1_node.html#a8a828c2e91a4aa2a9ab7b94554e4685b">nz::nodes::Node</a>.</p>

<p class="definition">Definition at line <a class="el" href="_nodes_8cu_source.html#l00165">165</a> of file <a class="el" href="_nodes_8cu_source.html">Nodes.cu</a>.</p>

</div>
</div>
<hr/>The documentation for this class was generated from the following files:<ul>
<li>D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/<a class="el" href="_nodes_8cuh_source.html">Nodes.cuh</a></li>
<li>D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/<a class="el" href="_nodes_8cu_source.html">Nodes.cu</a></li>
</ul>
</div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated by&#160;<a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.12.0
</small></address>
</div><!-- doc-content -->
</body>
</html>
